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(57) Abstract: A system and method for computer simulation of image-guided diagnostic and therapeutic procedures such as vas- 
cular catheterization, angioplasty, stent, coil and graft placement, embolotherapy and drug infusion therapy. In a preferred aspect, 
^5 system is configured to resemble a cardiovascular catheterization laboratoiy where interventional radiology procedures are per- 
formed. A first user (1) may interactively manipulate therapeutic catheters, guidcwircs and other medical devices (5) in real-time 
while viewing patient-specific medical image data (2) in a manner similar to that encountered in a clinical procedure. 
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SBMUIATION SYSTEM FOR IMAGES-GUIDED MEDICAL PROCEDURES 

5 Related Applications 

This application claims piioxity under 35 U.S.C. § 1 19(e) to U.S. Provisional Application 
60/273.733, and to U.S. Provisional Application Serial No. 60/273.734. both filed March 6. 2001. the 
entireties of which are incoiporated by reference herein. 

Field of the Inventiop 

10 The invention relates to a system and method for simulating image-guided medical 

procedures, particularly those relying on interventional and/or diagnostic devices such as catheters. 

Background of the Inventioa 

Interventional radiology provides an alternative to open field surgery. Interventional 
1 S radiology began as a discipline for diagnosing and treating vascular disease (e.g.. such as narrowing 
of the arteries). The interventional radiologist typically uses a catfieter inserted into a blood vessel 
tfuough a puncture in the skin to gain internal access to the vascular systeih. Using medical 
imaging guidance, the catheter is navigated to a target site (e.g., such as the site of a diseased 
tissue) where it can be used as a conduit through which to pass therapeutic devices. Minimally 
20 invasive procedures relying on interventional radiology, including lapraroscopic surgery, 

cardiovascular interventional radiology, and neurointerventional radiology, have tremendous 
potential to reduce patient discomfort, complications, hospital stays, post-procedural recovery time 
and total medical costs. 

However, despite the advantages described above, minimally invasive procedures also pose 
25 risks. Failure to properly navigate, orient or position catheters and/or other devices within a patient, 
or fciilure to properly recognize an anatomic area or pa&ology to be treated may result in serious 
injury to a vein, artery, organ, or other internal tissue structure. 

Interventional radiologists must perform the delicate eye-hand coordinated movements 
needed to navigate catheters and therapeutic devices while viewing scanned images of the patient's 
30 body cavities or lumens on a flat TV screen. The images are obtained from X-iays, CAT scans, 
MRI scans, and the like. Depth perception is lacking and it is difficult to learn to control the 
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ins truments thr ou gh the spatially arbitrary linkage. A mistake in this environment can be 

dangerous. Without performing the procedure often, there is no way for practitioners to maintain 
the high degree of skill needed to perform these procedures. It is also impractical to implement 
new methods, operations, and procedures on live individuals. 

5 U.S. Patent No. 6,038,488 discloses a catheter simulation device for surgery and 

interventional radiology procedures. Translation and rotation of the simulated catheter can be 
tracked and a computerized control system and recording device are employed to provide a 
programmed procedure which provides a realistic ''feel" to a user of a surgical procedure. 

WO 99/16352 describes using graphical representations of a surgical instrument and area of 
10 ^e body in which a procedure is being performed to aid an operator using the surgical instrument 

WO 99/393 1 5 describes a vascular access simulation system with a tracking system for 
monitoring the movements of a simulated catheter needle assembly and a skin traction mechanism. 
The system receives measurements from the tracking system to update simulation and display of 
representations of the catheter while providing control signals to a force feedback device to enable 
1 S the application of force to the catheter, needle assembly. 

WO 98/03954 describes a system for simulating operating conditions during minimally 
invasive surgical procedures. Data regarding the movements of a simulated surgical instrument are 
interpolated by a computer processor, which utilizes a database of information representing a 
patient's internal landscape to create a computer model of the internal landscape of the patient. 

20 U.S. Patent No, 6,106,301 describes a radiology interface apparatus and peripherals, such as 

mock medical instruments, for simulating performance of a medical procedure on a virtual patient 
The interface measures manipulations of system peripherals and transfers these measurements via a 
processor to a medical procedure simulation system. 

Summary of the Invention 

25 There is a need to have a highly realistic simulation environment for training and pretreatment 

plaiming of image>guided, medical procedures such as vascular catheterization, angioplasty, 
embolotherapy, drug infusion therapy and stent and coil deployment 

This invention is designed to assist physicians in their training and in preplanning of 
diagnostic and therapeutic procedures performed in the vascular cardiovascular catheterization 
30 laboratory. In addition to providing realistic visual feedback and interacting with essential devices, 
such as are found in a cardiovascular catheterization laboratory, the simulator also provides active 
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haptic force and tactile feedback components to enhance the total hand-eye coordinated experiences 
encountered by physicians during actual interventional procedures. To simulate various types of 
procedures, the invention also provides a novel solution to easily configure or customize ttie training 
or pretreatment planning environment to meet die needs of the user or trainer. 

5 In one aspect, the invention provides a system for simulating the movement of a medical 

device in a body cavity or lumen of a patient. The system comprises a medical device comprising a 
first end for manipulation by a user and a portion comprising a second end insertable into a simulated 
body cavity or body himen in a manikin. The manikin comprises an interface for receiving the 
portion comprising the second end and for interfacing with the simulated body cavity or lumen. The 
10 manikin furAer comprises a directional force feedback mechanism for exerting a directional force on 
the medical device in response to a feedback signal. 

Preferably, the directional force feedback mechanism provides resistance to forward motion 
but enables free reveise motion in response to the feedback signal. In one aspect, the directional force 
feedback mechanism comprises a rolling element coupled to tiie portion of tfie medical device 
15 comprising the second end. An internal surface of the simulated cavity or lumen in the manikin . 

comprises an oblique slot for receiving the rolling element. In response to a feedback signal, forward 
movement of the second end will cause the rolling element to be received by the slot, thereby causing 
resistance to fbr^er forward motion. A motor, such as a servo motor, can be used to control 
movement of the rolling element 

20 

In another aspect, the system further comprises a tactile feedback mechanism which can 
provide continuous vibrational feedback to a user holding the medical device. Tactile feedback can 
be provided through a continuously rotating motor in communication with the portion of the medical 
device comprising the second end. The tactile feedback mechanism can simulate such parameters as 
25 blood flow, respiration and the like. 

Preferably, the position of at least the second end of ttxe medical device relative the simulated 
body cavity or lumen is continuously tracked. In one aspect, the medical device comprises an 
encoder for tracking the translation of the device and an encoder for tracking the rotation of the 
device. In another aspect, the system comprises a tracking unit comprising a light source, a signal 
30 processing circuit, and one or more optical sensors which are placed within the interface. Light from 
the light source reflects on the device when it is inserted into the interface and reflected light from the 
device is received by the one or more optical sensors of the tracking system, enabling the movement 
of the device to be tracked. In response to detection of changes in reflected light, the system will 
simulate movement of the device in real-time on a user display. 
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In-one -aspectj-two-optical-sensors -are-provided which lie in two different planes which are 

perpendicular to each other. In another aspect, the tracking unit is configured in the form of a rail 
along which the device can move. 

More than one medical device can be inserted into the interface and the position of each 
medical device can be independently monitored. Suitable medical devices for use with the system 
include, but are not limited to: a catheter, guidewire, endoscope, laparoscope, bronchoscope, stent, 
coil, balloon, a balloon-inflating device, a surgical tool, a vascular occlusion device, optical probe, a 
drug delivery device, and combinations thereoC 

In one aspect, the system comprises a table on which the manikin is placed. The table can 
comprise a processor coimectable to the network. Additional ancillary equipment can include, but 
are not limited to, foot and hand activation switches, radioopaque contrast dye injectors, hand 
operated balloon inflation devices, and monitors displaying various simulated electrophysiological, 
parameters. 

The system comprises at least one first user device coiinec table to the network which 
comprises a first display inter&ce for displaying a three-dimensional representation of a simulated 
body cavity or lumen of a patient. Preferably, the first display interface further displays a three- 
dimensional representation of a medical device corresponding to a medical device which is interfaced 
with the manikin. The system simulates the movement of the medical device within the simulated 
body cavity or lumen of the manikin in real-time on the display, when a first user manipulates the 
medical device interfaced with the manikin. For example, die system can be used to simulate die 
movement of a catheter through a blood vessel (e.g., such as a blood vessel in the heart or brain or 
another organ). In one aspect, the system further simulates deformation of the body cavity or lumen 
by the medical device. 

The system also can comprise a simulated scarming display for displaying 
a two-dimensional image of the simulated body cavity or lumen. In one aspect, the scaiming display 
is part of a simulated scaiming device. For example, the simulated scanning device can simulate an 
ray imaging system. Both the simulated scanning device and scarming display can be coupled to a 
movable C-arm within scaiming distance of the manikin. 

To enhance the realism of the simulation, a re-configurable control panel (e.g., a touch screen) 
Can be provided for performing one or more of: image acquisition selection; image display; 
' manipulating a table on which the manikin is placed; manipulating the position of a simulated 
scanning device relative to the manikin; and control of one or more shutter devices for limiting a field 
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pf3dew-Q£a-ScaimiDg_deviceLj3lac&dLwithin scannmgjdistance-o£the manncin— The system-also-can 

comprise at least one foot-activation switch for activating or collimating the simulated scanning 
device, image display and/or for controlling table movement. 

The system can be ad^ted for use by multiple users^ for example, as part of a training 
5 environmenL In one aspect, the system comprises a monitoring station comprising a second user 
device connectable to the network and comprising a second display interface for enabling a second 
user to monitor the movement of die medical device within the simulated body cavity or lumen. 
Preferably, the second display inteiface displays selectable options (e»g., a drop down menu, action 
buttons, check buttons, radio buttons, dialog boxes, command lines and the like), enabling the second 
1 0 user to select or change one or more anatomical and/or physiological parameters of die simulated 
body cavity or lumen. Selection of a selectable option causes die three-dimensional image of the 
simulated body cavity or lumen displayed to the first user to change to reflect the changed anatomical 
and/or physiological parameteis selected by die second user. - r j-'* 

15 In one aspect, the first user display inter&ce provides access to a database and in response to * 

this accessing, the system displays an image and/or medical data of a patient on tiie- first user display - 
interface. In another aspect, the second user di^lay interface provides access to the database and the 
system displays an image and/or medical data on the second user display interface. In a further 
aspect, the second user display interface provides a selectable option enabling a second user to 

20 display the image which is displayed on die second user display interface, on the first user's display 
interface. i . . . 

The invention additionally provides a syringe for simulating fluid delivery, comprising: a 
housing defining a lumen comprising an opening for delivering a fluid, 
a pushing element for pushing the fluid through the opening, a friction-producing element in 
25 communication with the pushing element, and a motor in communication with the friction-producing 
element and comprising a signal-receiving element The friction-producing element will cause 
friction between the pushing element and a surface of the lumen of die housing when the motor is 
activated. Activation of the motor is responsive to a signal received by the signal-receiving element 

When activated, the motor causes motion of the friction-producing element, causing the 
30 fnction-producing element to contact the surface of the lumen of the housing. This creates friction 
between the pushing element and the surface of the lumen and causes resistance to the motion of the 
pushing element, thus simulating injection of a fluid through a syringe into the body of a patient 
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_ The friction-producing element can comprise one or more rubber pads, each rubber pad being 
coupled to an arm whose movement is controlled by the motor, e.g., such as throu^ a gear attached 
to the motor. In one aspect, the amount of friction produced by the friction-producing element is 
adjusted by controlling a rotation angle of the motor. 

The syringe can be used with tiie system described above. For example, the opening of the 
syringe can be coimectable to a coimecting piece having a first end for receiving fluid from the 
opening and a second end for delivering fluid to a simulated body cavity or body iuxhen in the 
manikin, e.g., via the manikin interface. 

The invention also provides a method for simulating fluid delivery into a body cavity or lumen 
of a patient comprising providing a syringe as described above and providing a signal to the syringe, 
thereby causing friction between the friction producing element and the pushing element Delivery of 
fluid to a body cavity or lumen can be viewed in real-time on the screen of a first and/or second user 
interface. 

. The invention also provides a balloon-inflating device for simulating deployment of a balloon 
within a body cavity or lumen of a patient In one aspect, the device comprises a delivery mechanism 
for controlling delivery of fluid (e.g., such as air) through the balloon-inflating device to the balloon, 
a pressure sensor for monitoring pressure of fluid delivered, and an electrical pressure meter for 
reading pressure determined by the pressure sensor. Preferably, the electrical pressure meter is 
connectable to a processor and can transmit a signal corresponding to a pressure value to the 
processor. The balloon inflating device can be used with the system; for example, the system can 
simulate the deployment of a balloon (e.g., inflation or deflation) within the lumen of a patient; i.e,, 
the system can be used to simulate a balloon angioplasty procedure. . 

The system also can be used to simulate other operations of medical devices, such as a 
surgical procedure (e.g., such as removal or repair of a tissue structure), injection of a radioopaque 
isotope and the like. 

The invention further provides methods for using the system described above. In one aspect, 
the invention provides a method for simulating the movement of a medical device in the body cavity 
or lumen of a patient, comprising: providing a medical device comprising a flrst end for manipulation 
by a user and a. portion comprising a second end inserted into a simulated body cavity or body lumen 
in a manikin, The simulated body cavity or lumen in the manikin comprises a directional force 
feedback mechanism and in response to a feedback signal, the directional force feedback mechanism 
creates resistance to forward motion of the medical device but allows fi^ee reverse motion, allowing a 



Best Available Copy 



WO02A)70980 ^ Paoe a of 71 



r WO.02/070980 




PCT/US02/06604 



first user to experience the sensation of using the device in the body of a patient In a preferre d 

aspect, the user experiences tactile feedback as well as directional feedback. 

The first user of the system can access a database of diree-dimensional ima^ of body 
5 cavities and lumens from a plurality of different patients. For example, the fiist user can request a 
rq^resentation of a selected image to be displayed on the di^lay interface of his or her user device in 
response to this accessing. For exan^le, the user can select a suitable hyperlink displayed on the 
interface, or can input a query into a command line or dialog box, or can select a selectable option 
provided on the interface in response to the user's accessing the database. 

10 In one aspect, the system also superin^oses a three-dimensional representation of the medical 

device on liie representation of the body cavity or lumen. Preferably, the system simulates the 
movement of the medical device within the body cavity or lumen in real-time on the display interface 
as the user manipulates the medical device which is interfaced with the manikin. More preferably, 
the system also simulates deformation of the body cavity or lumen as the first user simulates 

15 navigating and/or deploying the medical device. The user can experience one or more simulated 
operations of the device, such as a surgical prx>cedure, injection of a mdioopaque dye, balloon 
deployment, and the like. 

The first user can manipulate a plurality of medical devices using the interface of the manikin. 
In response, the system will simulate the movement of each of these devices on the first user's display 
20 interface. For example, the system can simulate ^e can simulate navigating a first catheter to a target 
region of the body, then a balloon catheter, and then positioning a balloon deployment device (e.g.,* 
such as the one described above) in proximity to the balloon catheter to inflate or deflate the balloon. 

In another aspect, the system also simulates inserting a stent catheter, navigating the stent 
catheter to the target region, and using the balloon deployed by the balloon catheter to deploy the 

25 stent In a further aspect, the system simulates coil embolization in a body cavity or lumen of a 

patient As a first user interacts with the manikin by inserting a catheter, guidewire, and coil wire into 
the simulated body cavity or lumen, the system simulates movement of each of these devices in the 
simulated body cavity or lumen on the first user's display interface. A selectable option on a control 
interface provided as part of the system (e.g., a re-configurable control panel or touch screen) 

30 ' provides the user with a selectable option for detaching the coil from the coil wire. When the user 

' selects the selectable option, the coil is released from the coil wire in the manikin and on the screen of 
tfie first user's display interface. 
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A second user also can interact with the first use r by using the monitoring station described 

above. For example, the second user can display a particular image selected by the second user from 
the database on the first user's display interface. The second user can alter parameters of the 
simulation displayed to the first user, for example, as part of a training exercise, to document flae 
5 progress of one or moie first users, and/or to introduce procedural variables that can be used to test 
or evaluate the response and decision-making abilities of one or more first users. 

In a particularly preferred aspect, the system is configured to resemble a cardiovascular 
catheterization laboratoxy where interventional radiology procedures are performed. A first user 
can interactively manipulate therapeutic catheters, guidewiies and o^er medical devices in real- 
10 time while viewing patient-specific medical image data sets in a manner similar to diat encountered 
in a clinical procedure. 

Brief Description of the Figures 

The objects and features of the invention can be better understood with reference to the 
following detailed description and accompanying drawings. 

15 Figure 1 is the block diagram of a simulation system according to one aspect of the invention. 

Figure 2 illustrates a perspective view of a system accoxding to the invention. A simulated 
patient (6) or manikin houses tracking and force feedback assemblies. Simulated medical devices are 
introduced into the simulated patient via a user interface box embedded within the patient/manikin 
(e.g.. through the groin or axillary area, in the case of a simulated catheterization lab). A first user (1) 
20 navigating and/or deploying simulated devices can be monitored by a second user (19) according to 
one aspect of the invention. 

Figure 3 is a block diagram showing various components of a system processor according to 
one aspect of tfie invention. 

Figure 4 is a block diagram showing various system inputs according one aspect of the 
25 invention, such as touch screens, footswitches, syringes, C-arm, hand-operated balloon device, 
feedback structures, and their connections with the system processor. 

Figure 5 A illustrates the mechanical structure of a tracking system and active force feedback 
structure according tb one aspect of the invention. More than one set of catheters and guidewiies can 
be used in a given simulation- The translation and rotation of each device are measured using 
30 incremental encoders. For example, encoder A and encoder B measure the translation and rotation of 
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a catheter , res pectively ^ Fi g ure SB is a bl ock diagra m illustratin g com ponents of the CrBcking system 

and active force feedback system and their interactions wiA the system processor. 

Figure 6A illustrates a different tracking system according to one aspect of the invention 
comprising one or more optical sensois. Figure 6B shows a tracking unit and catheter device 
5 configured as a loop which can be used to simulate pushing, pulling, or twisting. 

Figures 7A and B show more detailed views of a directional force feedback mechanism 
according to one aspect of the invention. As shown in Figure 7A, a wheel coupled to a motor directs 
forward movement of a simulated catheter through a shutter connected to the simulate catheter. 
Forward and downward motion of the shutter, forces a rolling element coupled to the catheter into an 
10 oblique slot which creates resistance to further forward motion. A discontinuously rotating hand 

provides a sensation of vibration. Figure 7B shows a close up of the rolling element (e.g., a shafi) and 
a cross-section througih the oblique slot 

Figures 8A1 -3 and B shows the structure of a simulated syringe with a force feedback 
mechanism according to one aspiect of the inverntion. Figure 8A shows a cross-section through the . 
1 5 longitudinal axis of the device while Figure 8B shows a cross-section through the tranverse axis of 



Figure 9 shows a simulated hand-operated balloon-inflating device according to one aspect of 
the invention. A pressure sensor is used to measure the pressure of inflation of a balloon attached to a 
catheter. The sensor signal is processed and transmitted to the system processor. A user can read the 
20 pressure value from a display monitor or fiom a piezometer as shown in the upper portion of the 

Figure. A user also can feel pressure transmitted back frorri a catheter port from ihc balloon during 
the inflation process. 

Figures lOA-D show re-configurable control panels according to different aspects of the 
invention. 

25 Figure 1 1 shows a scaled-down simulation system according to one aspect of the invention. 

Figure 12 shows a desktop simulation system for pretreatment planning comprising a re- 
configurable control panel. 



die device. 



Figure 13 is a flow chart showing the steps of a simulation process according to one aspect of 
the invention. 
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EigutieJ^_illus.trates_the.time/realism.requirements.of-a.simulation^ to the 



invention. 



Figure 15 shows a physical model of vascular data and device data which can be ixq^utted into 
a system processor according to the invention. 

Figure 16 illustrates creation of a physical model of a body cavity or blood vessel according to 
one aspect of the invention. 



Detailed Description 

The invention provides a system for the simulation of image-guided medical procedures and 
10 mediods of using the same. The system can be used for training and certification, pre-treatment 
planning, as well therapeutic device design, development and evaluation. 

Definitions 

The following definitions are provided for specific terms which are used in die following 
written description. 

15 As used herein, "coupled to" refers to direct or indirect coupling of one element of a system to 

another. An element may be removably coupled or permanently coupled to another element of the 
system. 

As used herein, "within scanning distance** refers to a distance which is close enough to the 
manikin to pemiit display of an image of the simulated body cavity or lumen on die scanning display 
20 of the system. 

As used herein, •*a re-configurable control panel** refers to a display inter&ce comprising one 
or more selectable options (e.g., in the form of action buttons, radio buttons, check buttons, drop- 
down menus, and the like) which can be selected by a user and which can direct the system to 
perform operation(s). Preferably, the one or more options can be selected by touch. The control 
25 panel can be modified by a user (e.g., by implementing a system program which alters the display, 
causing it to display different selectable options) thereby "re-configuring" the control panel. 

As used herein, •*providing access to a database** refers to providing a selectable option on the 
display of a user device which, when selected, causes the system to display images or data stored 
within the database, or causes one or more links to be displayed which, when selected, causes the 
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images or data, in response to a queiy of the system by a user. In one aspect, the display interface 
provides a "queiy input field" into which the user can input a query and the selectable option is an 
action button for transmitting the queiy to the system. 

5 As used herein, the term "in communication with" refers to the ability of a system or 

component of a system to receive input data from another system or component of a system and to 
provide an output response in response to the input data. "Ou^uf * may be in the form of data or may 
be in the form of an action takeii by ftie system or component of &e system. 

As used herein, "deployment of a balloon" refers to either inflation or deflation of the balloon. 

10 As used herein, a pathology "affecting the structure of &e body cavity or lumen" is one which 

measurably alters at least one physical property of the body cavity or lumen... 

As used herein, "a physical property" refers to a property which relates to the structure or 
anatomy of a body cavity or lumen which is measumble, generally without the aid of a labeled 
molecular probe; for example, physical properties of a blood vessel include, but are not limited to: 
15 elasticity^ thickness, strength of ventricular contractions, vascular resistance, fluid volume, cardiac 
ou^ut, myocardial contractility, and other related parameters. 

As used herein, "a volume image" is a stack of two-dimensional (2D) ima^s (e.g., of a body 
cavity or lumen) oriented in an axial direction. 

As used herein, a device for "accessing a body cavity or lumen" refers to a device which can 
20 be maneuvered in the body cavity or lumen. "Maneuvering^* refers to the ability of at least about 50% 
of the external surface of the device to fit within a cavity or lumen ^^ile retaining rotational or 
forward translational freedom of movement. 

As used herein, an "interventional medical device" includes a device for treatment (e.g., stents, 
stent-grafts, balloons, coils, drug delivery devices), for diagnosis (e.g., imaging probes), and for 
25 placement of other medical devices (e.g., guidewires). Some devices, such as catheters, can have 

multiple functions. In general, the terms "an interventional medical device" and "device for accessing 
a body cavity or lumen" are used interchangeably. 

As used herein, a "knowledge base " is a data structure comprising facts and rules relating to a 
subject; for example, a "vascular properties knowledge base" is a data structure comprising facts 
30 relating to properties of blood vessels, such as elasticity, deformation, tissue and cellular properties. 
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blood fl ow^ and the like and rules for correlatin g f acts relating in vagf^nlqr prpp^rtigs tf> mteractions 
widi one or medical devices. 

As used herein, a "rule" in a knowledge base refers to a statement associated with a certainty 
^tor. Rules are geneially established by interviewing human experts or by obtaining data from 
5 databases or other knowledge bases. 



engine which enables the system to manipulate input data from a user to arrive at one or more 
1 0 possible answers to a question by a user. More preferably, an expert system also comprises a cache 
or dynamic memory for storing the current state of any active rule along with facts relating to 



As used herein, a system which "simulates a path representing at least a portion of a body 
cavity or lumen*' is a system which displays a three-dimensional representation of the internal surface 
1 5 of the at least a portion of the body cavity or lumen on the interface of a user device in 
communication with tfie system. 

As used herein, to "determine the best fit between the geometry of the device and the 
geometry of the path" refers to displaying a representation of at least a portion of the device and 
simulating its placement within at least a portion of the body cavity or lumen. • 

20 As used herein, a "device parameter" refers to a physical property of a device, e.g., such as 

flexibility, memory, material, shape, and the like. 

As used herein, "a physical model of a device" is a combination of a reconmiended 
geometrical model, topology, and material. It is also the basis for making the first design of a 
medical device based on patient-specific data. 

25 As used herein, a "software suite" refers to a plurality of interacting programs for 

communicating with an operating system. 

As used herein, "clinical data" refeis to physical, anatomical, and/or physiological data 
acquired by medical image modalities such as X-ray, MRI, CT, US, angiography, video camera, or by 
direct .physical and/or electronic measurements. 



As used herein, an "expert system" comprises a program for applying the rules of one or more 
knowledge bases to data provided to, or stored within the knowledge ba5e(s), thereby enabling the 
knowledge base(s) to be queried and to grow. Preferably, an expert system comprises an inference 



premises on which the rule is based. 
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As used herein, "a best fit" between a simulated path for a simulated body cavity or lumen and 
a simulated medical device refers to one which requires the minimum amount of deformation in the 
simulated surgical process that takes into consideration the patient-specific vasculature and composite 
materials of tiie device. 

5 As used herein, an "FEM engine" refers to a program or set of programs for performing finite 

element analysis focusing on vasculature finite element models and analysis of the interaction 
between vasculature models and devices. 

Intervention Simulation System 

Figure 1 is a block diagram of a simulation system according to one aspect of the invention. 

10 Input into the system executes a particular simulation to be enacted. Generally, a simulation includes 
images of a patient and also can include a display of patient-specific information (e.g., such as clinical 
information and medical history). The patient images can be obtained from a database of patient- 
specific images or images relating to a population of demographically similar patients (e.g., such as 
patients sharing a pathology). Preferably, the system also accesses data relating to various 

15 interventional devices. For example, the system can include data files relating to the sh^e and 

physical properties of one or more medical devices. Preferably, the simulation system also includes a 
manikin (6) (i.e., representing a patient) for interfacing wifli one or more siniulated medical devices 
(see, e.g., as shown in Figure 2). 

Users interact with the simulation system by observing various rendered displays (2), such as 
20 fluoroscopic images of blood vessels, and manipulate the one or more simulated devices in response- 
to data received by these display (2). The invention can simulate the anatomy of a specific patient on 
which tiie one or more devices will be used. 

Simulated Patient Interface 

In one aspect, the intervention simulation system comprises a medical device with a first end 
2S for manipulation by a user and a portion comprising a second end which is insertable into a simulated 
body cavity or body lumen in a simulated patient interface. Preferably, the simulated patient interfece 
is part of a manikin (6) which simulates the physical features of a human patient. The manikin (6) 
comprises an interface (5) for receiving the portion comprising the second end and for interfacing 
with a simulated body cavity or lumen within the manikin. 



30 



The interface also comprises a directional force feedback mechanism for exerting a 
directional force on the candidate medical device in response to a feedback signal received by &e 
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force feedback mechapism. This provides a user with a feeling that heyshe is interacting with _a real 
patient Preferably, the directional force feedback mechanism provides resistance to forward motion 
but enables reverse motion in response to the feedback signal. The medical device can be one which 
is commercially available, a device being tested for use or not otherwise commercially available, or 
can be a candidate device based on a simulated design selected to suit die char^teristics of a 
particular patient Methods and systems for designing customized medical devices are disclosed in 
U.S. Provisional Application Serial No. 60/273,734, filed March 6, 2001. 

As shown in Figures 7A and B, in one aspect, the directional force feedback mechanism 
comprises a rolling element (e.g., such as a shaft) coupled to the portion of the candidate device 
comprising the second end. An internal surface of the simulated cavity or lumen in the manikin in 
turn comprises an oblique slot for receiving the rolling element In response to a feedback signal, 
forward movement of the second end causes the rolling element to be received by the slot, causing 
resistance to furflier forward motion. Preferably, a motor, such as a servo motor, controls movement 
of the rolling element This embodiment is shown in more detail in. Figure 7 A. When a force 
feedback control signal is generated, the servo motor rotates clockwise, causing a wheel to rotate, 
pushing a shutter forward and downward. The forward and downward movement of die shutter ' 
pushes the rolling element down into the slot. Because of friction between the slot and tiie rolling 
element, the rolling element moves upward along the slot, and any further forward movement is 
resisted. If the user wishes to pull back the device, the rolling element will move downward and can 
be released from the slot 

In a most preferred embodiment, the interface comprises bodi a directional force feedback 
mechanism and a tactile feedback mechanism. The tactile feedback mechanism provides vibrational 
feedback to a user holding the candidate medical device. In one aspect, continuous vibrational 
feedback is provided throu^ a continuously rotating motor (e.g., such as die servo motor) in 
communication with the portion of the device comprising the second ^d. A discontinuously rotating 
motor controlling the movement of a hand (see, e.g.. Figure 7 A) also can be used to simulate 
intermittent forces experienced by a body cavity or lumen. For exarhple, in a clinical situation, when 
a catheter is inserted into the cardiovascular system, cardiovascular function and activity may cause 
the catheter to vibrate. The servo motor and the rotating hand are designed to simulate this kind of 
vibration. The servo motor rotates continuously and but has an adjustable rotating frequency. . The 
rotating hand pushes the catheter discontinuously, i.e., simulating blood flow and respiration. 



Haptic "display" serves at least two purposes in the intervention simulator: kinesthetic and 
cognitive. First, it provides the sensation of movement to the user and therefore it greaUy enhances 
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sur gical performance. Second, it is used to enable a user to leam to d istinguish between tissues by 

testing their mechanical properties. Deformation models are used to compute the effect of force on 
the biomechanics of an interventional device and on the deformation of a cavity or himen through 
which the de\dce is navigating and/or being deployed, and feedback signals to the force feedback and 
5 tactile feedback mechanisms can be used to mimic actual forces ^at might be experienced during an 
interventional procedure. 

However, although tfie system is capable of providing physically meaningful forces that 
realistically mimic those encountered during an interventional procedure, there may be instances 
where a less lealistic simulation is satisfactory. For example, a user may want to feel that he or she 
10 has reached an obstmction during catheterization. However, the user may not need to experience all 
the various contacts he or she could feel while maneuvering a catheter prior to experiencing a singile 
large resistance. The present invention provides a mechanism to stop a user from advancing past an 
obstruction and yet p^ermits the user to pull back a simulated device.. 

The manikin interface can be encased in a housing comprising one or more openings for 
15 receiving medical devices, and means for interfacing with ttacking unit(s), feedback mechanism(s) 
and a system processor (described further below). Additional devices such as syringes and balloon 
inflating devices can be provided as part of the interface, e.g., simulating balloon angioplasty 
proceedings). 

The interface housing can be displaceable for some distance from the manikin itself or can 
20 project from the manikin (e.g., being an integral part of the manikin). To further enhance realism, 

only the opening(s) of the housing may visible from the manikin (e.g., the interface *Tiousing*'.can be 
part of the manikin). Preferably, the interface is an embedded system as shown in Figure 2, with 
openings into areas of the manikin simulating areas of medical intervention. 

System Input Devices . 

25 The simulation system can obtain input of various types to more closely mimic an intervention 

procedure. For example, as shown in Figure 1, input to the simulator can consist of patient medical 
history and diagnostic data including, but not limited to, data obtained from X-ray, MRI, MRA, CT or 
ultrasound images. Data can relate to a specific patient, e.g., where a user is training to perform a 
procedure on a specific patient using a customized device. Alternatively, data can relate to a 

30 "symbolic patient", for example, representing a particular demographic group of closely related 
patients, such as patients having a type of pathology. 
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Figure 2 shows an example of various system inputs that can be provided. The simulated 
patient (6) in the Figure is a manikin that also houses the tracking and force feedback assemblies. A 
physician (1) simulates navigation of a catheter by manipulating catheteis and guidewiies (5). These 
ca&eters and guide wires are inserted into fhc manikin at the manikin inter&ce. One or more monitors 
5 (2) can be used to display simulated fluoroscopic and vascular images simulating the internal 
anatomy of a patient represented by the manikin. In one aspect, 2-D fluoroscopic views are 
displayed at the same time that 3D geometric models are displayed by system user interfaces. 
Preferably, the user has the option to adjust fluoroscopic images by one or more of zooming, 
collimation, rotation, and the like. In combination with 3D volume-rendered images generated using 
10 display interfaces described further below, a user can view the vasculature from various positions or 
angles along x-, y-, and x- axes. This option can be of major value in pre-treatment planning, since a 
physician can use the system to evaluate different treatment approaches prior to performing actual 
intervention in a patient 

A simulated scanning device (4) additionally can be provided, e.g., in the form of a mock C- 
15 arm equipped with an x-ray emitter. Preferably, the mock C~arm can move along the long side of 
an operating table (8) on which the patient/manikin is placed and can rotate around the table to 
simulate capturing a patient's images at various lateral and angular positions. 

For example, as shown in Figure 2, two footswitches (7) can be used- to simulate activation of 
a simulated x-ray device as well as vascular image acquisition and storage. In response to this 
20 activation, one or more monitors Qt) simulate fluoroscopic images obtained. A footswitch is' 

preferred for scanning and image processing, since user(s) generally have their hands occupied with 
other equipment, in actual practice 

Preferably, the system provides a reKx>nfigurabIe control panel (9) (e.g., a touch screen) to 
enable a user to simulate patient table manipulation, vascular image acquisition selection and display, 
25 and the use of shutter devices to limit the extent of the field of view provided by a scaiming device 
(4). The panel also can be used to implement functions such as machine-activated radioopaque dye 
injection (e.g., activating the simulated syringe) and/or deployment of a balloon via a balloon- 
inflating device (18) interfaced with the manikin. 



30 invention. Preferably, the display is programmable and has a large storage area for bitmaps, display 
lists, and screens^ Users can easily set up complex image control panels according to their own 
requirements. Figure IDA shows the typical controls associated with patient table manipulation. 
Manipulating these controls result in a corresponding motion of a C-arm that carries a simulated X- 



Figures IDA — D show re-configurable control panels according to different aspects of the 
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ray emitter. Figure lOB shows controls associated with ima g e acquisition for mani pulating the 
sharpness and clarity of the rendered images. Figure IOC shows controls for machine activated 
balloon inflation and deflation, and shuttering. Figure lOD shows controls for collecting images for 



interfaced with the manikin for simulating common procedures such as an injection, balloon 
deployment, and/or stent deployment 

In one aspect, therefore, the system comprises a syringe for simulating fluid delivery. The 
simulation syringe comprises a housing defining a lumen comprising an opening for delivering a 
10 fluid, a pushing element for pushing the fluid through the opening, a friction-producing element in 

communication with the pushing element, and a motor in communication with the friction-producing 
element which further comprises signal receiving element. The simulation S3ainge can be interfaced 
with the manikin and the signal-receiving element preferably receives signals from the system 
processor, e.g., to execute contrast injection at a particular flow rate and volume. 

15 The firiction-producing element will cause friction between the pushing element and a surface 

of the lumen of the housing when the motor is activated in response to a signal received by the signal- 
receiving element When activated, flie motor causes motion of the fnction-producing element, 
causing the friction-producing element to contact the surface of the lumen of the housing. This 
creates friction between the pushing element and the surface of the lumen and causes resistance to the 

20 motion of the pushing element, thus simulating injection of a fluid through a syringe into the body of 
a patient 

The friction-producing element can comprise one or more rubber pads, each rubber pad being 
coupled to an arm whose movement is controlled by the motor, eig., such as through a gear attached 
to the motor. In one aspect, the amount of friction produced by the friction-producing element is 
25 adjusted by controlling a rotation angle of the motor. 

Figures 8A1 -3 and B show the structure of a simulated syringe (3) with force feedback 
stmcture according to one aspect of the invention. In the embodiment shown in Figures 8A-B, a 
servo motor and two arms are installed in front of the pushing element (e.g., a handspike). These two 
arms are connected through two meshed gears. Gear 1 is installed on the servo motor as is the driver 
30 of the motor. When a force feedback signal is communicated to the servo motor, the Gear 1 will 

contrarotate and Gear2 will rotate clockwise. Arm 1 and Arm 2 splay and the rubber pads on the two 
arms will touch the wall of the syringe. Because of the friction, the surgeon will feel resistance when 



creating a roadmap. 



5 



In a preferred aspect, the invention additionally provides one or more interventional devices 
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he_er_she_taesJo_push.oiLpullJthe^pushing.element by 

controlling the rotation angle of the servo motor. When the servo motor rotates clockwise, Ae two 
arms will close and the user can move the handspike freely again. 

The simulated syringe (3) can be used to simulate the mjection of radioopaque dye, to 
5 simulate fluoroscopic imaging, or the delivery of a therapeutic agent or drug. Control parameters 

such as contrast injection volume and rate can be controlled by a user through a control interface such 
as a touch screen, enabling a user to choose the rate and total volume of injection. The injection 
process can be captured, and selected images of ftie process saved, to provide a roadmap image on a 
separate monitor. 

10 As discussed above, the manikin also can be interfaced with a balloon-inflating device for 

simulating such operations as balloon and stent deployment Figure 9 shows a simulated hand- 
operated balloon-inflating device. A pressure sensor is used to measure the pressure of a fluid, such 
as air, being delivered to a balloon catheter, which has been navigated to a target site. The signal of 
the sensor is processed and transmitted to the system processor through the microprocessor Ml. A 

15 user is able to read pressure values from a display monitor or ftora an electrical pressure sensor, such 
as the piezometer shown in the upper portion of Figure 9. Additionally, the user is able to feel 
pressure delivered by the balloon-inflating device. Preferably, the inflating device can provide up tO 
about at least 20 Bar of compressed liquid to a balloon. 

In one aspect, compressed liquid is provided widi in a compressed liquid generator in 
20 communication with a piston. A user can trigger delivery of compressed fluid firom the generator by 
imparting linear force on the piston, thereby triggering inflation of a balloon. Preferably, the 
compression stroke is approximately 2 inches and can be delivered when a user twists a handle of the 
balloon-inflating device, causing a screw bar to force the piston forward. A release button or switch 
is provided enable rapid release of compressed fluid, thereby triggering deflation of the balloon. In a 
25 further aspect, an automatic control system inside the device is used to control the forward or 

backward movement of a handwheel for controlling the amount of pressure actually delivered to the 
balloon, in steps of a minimum of 0.01 Bar. Pressure values can be read from a screen of the pressure 
meter (e.g., such as an LCD screen) or can be displayed on the display of a user device. 

Preferably, the system enables a user to program parameters such as maximum pressure 
30 delivered to the balloon. In one aspect, the user starts to increase pressure delivered to the balloon by 
means. of a button on a handheld remote control and monitors pressure values on a pressure display. 
The system may trigger an alarm when balloon pressure increases past a selected threshold. In an 
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"auto mode" the system analyzes pressure data automatically to automatically adjust pressure and 
increase/decrease speed of inflation or deflation as appropriate. 



5 



Signals from simulated devices such as the simulated syringe and/or hand operated balloon 
inflation device generally are processed by an A/D converter first and then inputted to the system 
processor. Preferably, signals from footswitches are digital and inputted directly to tfie 
microprocessor and then to the system processor. 



The system can include additional input devices to simulate an interventional procedure. For 
example, one or more monitors can be included to simulate display of dlectrophysiological signals 
such as ECG and blood pressure. 



10 



The system is designed to allow use by multiple users (see, e.g.. Figure 2). For example, a 
second user (19) can be introduced to alter the simulation parameters that a first user (1) is 
experiencing. In one aspect, therefore, the system further comprises one or more monitors (13-17) 



comprising one or more second user display interfeces for a enabling a second user (e.g., a trainer) 
(19) to monitor a simulation that a first user (I) is experiencing. The second user/trainer ( 1 9) is 
15 provided with selectable options on the display of his or her user interface to enable die second user 
(19) to alter or introduce variables (e.g., anatomical or physiological variables) in order to test or 
evaluate the responses or decision-making abilities of one or more first users (1). 

In one aspect, processor interlace (13) is used to connect a monitor (15), keyboard (16) and 
mouse (1 7) of a first user device to a system processor connectable to the network. In one aspect, the . 

20 system processor communicates with a microprocessor (Ml) which resides inside an operating table 
which receives the manikin. Using a floppy disk driver (10) and/or CD-ROM drive (12), and/or other 
memory devices, new patient cases can be loaded into the simulation system. Because the processor 
is connectable to the internet or intranet, remote observation or training is possible. A user (1 or 19) 
can upload relevant patient images or other medical data or can upgrade software using tfie monitor 

25 (1 5), keyboard (1 6) and mouse (1 7). 

Figure 3 is a block diagram highlighting the features of the system processor, according to one 
aspect of the invention. In the aspect shown in Figure 2, the processor is a CPU installed inside a 
patient table (8) which controls data and control flow among various system con:^onents such as a 
hard drive, memory, display monitors, manikin interface, and one or more simulated medical devices. 

30 Figure 4 is a block diagram showing interactions between the system processor and various 

system inputs according to one aspect of the invention. A microprocessor (Ml) is used for data 
acquisition. Tracking, information from an optical tracking unit in optical communication with one or 
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more simulated medical devices is pr ovide d to the Ml through a signal processing cirouit. A mock 

C-Axm moves in response to a control sigr^al from a touch screen connected to the microprocessor 
through a serial port. Preferably, the control signal is processed by the microprocessor prior to being 
received by the C-arm. 

5 Preferably, roanq^ulations of a device by a user in an interface provided in the manikin are 

coordinated with a simulation of the device on the user display, such that motion of the device in &e 
manikin is simulated on the user display in real-time. 

Therefore, in one aspect, the simulation device provides a mechanism to continuously track a 
position of at least the second end of the medical device relative to the manikin. For exEunple, the 

10 system can comprise one or more encoders for tracking translation and/or rotation of the device (see, 
e.g.. Figures 5 A and B). Figure SB is a block diagram illustrating the components of the tracking 
system and the active force feedback mechanism. Using rolling spherical objects coupled to a 
simulated catheter and guidewire, respectively, four incremental encoders can pick up the motions of 
the simulated devices. The rolling elements also serve as part of the directional force feedback 

15 mechanism described above. 

In a currently preferred aspect, as shown in Figures 6A and B, the system comprises a tracking 
unit which tracks the movement of the candidate medical device. The tracking system comprises a 
light source (e.g., a point light source), a signal processing circuit, and one or more optical sensors, 
and is placed within the interface in optical communication with the device and the simulated cavity 
20 or lumen through which the device is being navigated. The candidate device will reflect light to it 
from the light source and reflected light will be received by the optical sensor(s). Changes in 
reflected light picked up from the sensor(s) indicate movemient of the candidate device as a result of 
manipulation. 

In one aspect, two optical sensors are provided within the tracking unit, each perpendicular to 
25 die other. 

As shown in Figure 6A, the tracking unit can be in the form of a rail along which the device 
can move. Alternatively, the simulated interventional device can be looped around the tracking 
device and can be manipulated by pushing, pulling, and/or twisting the loop. Preferably, tracking 
systems enable the intervention system to independently track the movement of two or more medical 
30 devices, for example, a catheter and guidewire. 



Being an embedded system with detachable components, the simulation system is flexible and 
can be easily configured to suit the customized needs of its users. Figure 1 1 shows a possible 
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configuration that reduces the size of simulation s ystem to one w hich ca n be plac ed on a normal 

desktop. Figure 12 is a desktop system intended for pretreatment planning by interventional 
radiologists. 

5 System Processor 

The intervention simulation system further comprises at least one first user device (e.g., a 
computer or wireless device connectable to the network) comprising a graphical user interface and 
connectable to a system processor and/or the network. Preferably, the processor comprises one or 
more programs for generating a geometric model of a body cavity or lumen of a patient from stored 
10 or collected volume-rendered images obtained from one or more patients. In one aspect, the user can 
access a database of such images using the user interface, either by inputting text into a conmiand box 
or dialog box and implementing a search function of the system or by selecting one or more selectable 
options reflecting files and/or images stored in the database (e.g., a hyperlink designated by the name 
of the file). 

15 The system generally operates by means of a software suite that operates on a general purpose 

computer such as a PC or IBM-compatible device. Preferably, the system comprises a processor 
(e.g., as CPU), memory, graphics adaptor, printer controller, hard disk and controller, mouse 
controller, and the like. The processor should comprise a minimum of about 8 MB of RAM. 
Preferably, the frrst user display interface is part of a monitor which is cormected to a keybosud, 

20 mouse, and, optionally, printer and/or scanning device. The software suite of the system comprises a 
program (e.g., a C language program) that controls the system's user interface and data files, e.g., 
providing one or more of search functions, computation functions, and relationship-determining 
functions. In a preferred aspect, the system comprises a geometric modeling system for modeling a 
three-dimensional representation of a body lumen or cavity; a device modeling system for modeling a 

25 three-dimensional representation of one or more medical dev ices, and at least one knowledge base 
for modeling interactions between a simulated body cavity or lumen and a simulated medical device. 



30 directly on one or more user interfaces or can be stored in a system database as described above. 

Because the system user devices and processors are connectable to the network, patient data also can 
be accessed from remote databases. 



Geometric Modeling System 



In one aspect, optical data relating to the internal contours of a body cavity or lumen are 
obtained and provided to the intervention simulation system. The optical data can be displayed 
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To generate a volume-image, a stack of two-dimensional (2D) images is collected by a 
scamiing device in an axial direction and is used to form a three-dimensional (3D) structure (see, e.g«, 
as shown in Figures 3 A and 3B). Almost all medical scanners can produce these axial images or can 
5 produce images that can be converted easfly to axial images. Suitable scanning devices include, but 
are not limited to, x-ray devices, magnetic resonance imaging (MRI) devices, ultrasound C^S) 
devices, computerized tomography (CT) devices, rotational angiography devices, gadolinium- 
enhanced MR angiograph devices, or other ima^ng modalities. For example, rotational CT scanners 
capture patient data in the form of projection images. By using a Filtered Back Projection technique 
10 or Arithmetic Reconstmction Technique (ART), volumetric images can be constmcted. 

The system may be directly connected to the output of one or more scaiuiing devices, e.g., 
collecting optical data from such devices as these are acquired. However, in another aspect, the 
system may include a means for extracting features from individual scanned images (e.g., 
conununicated to the system through a scanner or provided as a pdf file) to construct a 3D volume 
1 5 image. The geometric modeling arm of the system can be implemented remotely by a user to 

determine one or more of: the geometry/topology of the body cavity or lumen, measurements relating 
to any pathological features of the body cavity or lumen, and such parameters as tissue wall thickness, 
elasticity and die like. 

In creating a geometric model of a body cavity or lumen (e.g., such as a blood vessel), a user 
20 of the system (e.g., a biomedical professional with knowledge of human anatomy and pathology) 

performs image processing tasks on a plurality of scanned images to create geometrical stmctures and 
a topology which corresponds to the contours of a body cavity or lumen belonging to a patient being 
analyzed. 

Volume rendering techniques such as ray casting and projection techniques have traditionally 
25 been used in the visualization of volume images, Ray casting methods shoot rays through a volume 
object from each pixel in an image and employ algoriduns that trilinearly interpolates samples along 
each ray, providing complex shading calculations and color assignments at the sample points which 
are then accumulated into final pixel colors (see, e.g., Kaufman, In Volume Renderings IEEE 
Computer Science Press, Las Alamitos, CA, 1990). Real-time volume rendering with hardware 
30 texture mapping (e.g., SGI) for UNIX platform or with board card (e.g., Mitsubishi VolumePro) for 
PC platforms are commercially available. 

Conunercially available image processing tools, such as Photoshop™ can be used to manually 
draw out the shape of the structure from each scanned image. Various imaging-processing tasks, as 



wo 02^)70980 



Page 24 of 71 




• 



PCT/US02/06604 



WO 02/070980 



23 



5 



are known in the art, can be perfonned by the system; for example, segmentation can be used. 
Several improved algorithms using iso^surfacing or volume-rendering techniques to visualize 
vascular trees also can be used and have been described in Ehiicke^ et al.. Computer &. Graphics 
18(3h 395-406, 1994; Cline, et Magnetic Resonance Imaging {Pergjainon Press) 7i 45-54, 

1989; and Puig, et al., Proc. Of Visualization '97, pp 443-446, for example. 



Projection-originated methods reconstruct 3D geometries from two or more images (See, e.g., 
Solbach, et al.. Computer Biomedical Researcit 27(3h 178-198, 1994; Nguyen and Sklansky, IEEE 
Transactions on Medical Imaging ISiS}:. 178-198, 1 994: Longuet-Higgens^ Nature 293n0h 133-135, 
1981). Thinning methods such as "active-contour", "medial axis transformation", and "simulated 

10 annealing**, and the like, can be employed to detennine inibimation in projection planes (see, e.g., 
Kass, et al.. International Journal of Computer Vision I: 32 1 -33 1 , 1 987; Lee, et al., CVGIP: 
Graphical Models and Image Processing 56f6) : 462-478, 1994; Arcelli and di Baja, Image and 
Vision Computing 11 ah 163-173, 1993; Pellot, et al., IEEE Transactions Medical Imaging 130) : 48- 
60, 1994; Brandt and Algazi, CVGIP: Image Understanding 55j[3}z 329-337, 1992). The advantage 

15 of projective reconstruction lies in its capability to handle tiny tube-like systems such as vascular, 
neural and lymphatic vessels that could be lost with iso-surfacing algorithms. 

"Piece-by-piece cylinder representation" or "generalized cylinder representation" is widely 
used in vascular modeling (see, e.g.. Brown et al.. Proceedings of EUROGRAPHY '87 ^ pp 1 13- 
124;Barillot et al., IEEE Transactions on Computer Graphics and Applications^ December 1985, pp 
20 13-19). Polygonal tessellation, e.g., triangulation, also can be applied to model 3D tube-like shapes 
as is known in the art (see, e.g., Sederberg, et al., IntemationalJoumal on Computational Geometry 
and Applications 8C4k 389^06; Choi and Paric, Visual Computer 372-387, 1994). Ferley, et al.. 
Computer Graphics Forum 163 {5) : 283-293, 1997, additionally describes an implicit surface method 
for reconstruction of branching shapes. 

25 Accurate modeling of a 3D vascular network relies on good representations of vascular 

segments and bifurcations. Ideally, a vascular model should be visuaUy smooth and the detail of the 
display should be adaptable to fit application requirements. In one aspect, a constructive approach is 
used to model visually smooth vascular networks. In this approach, vascular segments are modeled 
using sweeping operations while vascular bifurcations can be modeled using blending operations (Le., 

30 sweeping plus hole- filling operations) (Gregory and Zhou, Computer Aided Geometric Design IJi 
391-410, 1994; Ye, et al.. Computer Aided Geometric Desigft 27: 875-885, 1995). Based on GC 
conditions for boundaries and cross-boundary derivatives (see, e.g., Schreiner and Buxbaum, IEEE 
Transactions on Biomedical Engineering 40Q): 482-491, 1993), constructive algorithms for 
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segmental sweeping and bifurcation blending can be designed as described in Cai et aL, 
"Constnictive Algoridims for GCl Generation of Vascular Network," Submitted to BEEE Biomedical 
Engineering, March 2001 . See, as shown in Figures 5 and 6. 

Yet ano^er method of obtaining a volume model or a geometric model of a body cavity or 
5 lumen is the technique of volumetric meshing. Meshes which represent a 3D or volumetric form can 
be generated from scanned images using a standard Windows operating system such as NT. Software 
for generating 3D mesh images are commercially and publicly available. Sources for such software 
are described at http://www-users.informatik.rwth»aachen.deAMPoberts/software.html^ 
and include, for example, A1taiT<S>HyperMesh<i)5.0 (available from Altair Engineering, Inc., 
10 Maplelawn, Troy, MI 48084). 

In a preferred aspect, a volume image of a blood vessel is obtained to construct a physical or 
geometric model which comprises information relating to both shape and material of tissue forming 
flie blood vessel Generally, construction of geometric models entail dividing a 3D modeling process 
into a series of 2D cross-sectional segmentation operations from which the 3D surface of the structure 

15 is reconstructed. Ah image processor is used to draw out the shape of the desired vascular structure 
from each image. Segmentation methods relying on intensity thresholding or region-growing can be 
used, as are known in the art, to facilitate the process (see, e.g., Wang, et aL, IEEE Engineering in 
Medicine and Biology, November/December 1 999, pp 33-39; Moore, et al., y. Biomechanics 57 : 1 79- 
184, 1998),. Finite element modeling also can be used for blood flow modeling, as described in, for 

20 example, Taylor, et aL, Computer Methods in Applied Mechanics and Engineering £58: 155-196; 
Hughes et aL, Computer Methods in Applied Mechanics and Engineering ZU^l- 173-189, 1989; 
Shephard and Georges, Int J. Numerical Methods in Engineering 32: 709-749, 1991. 

Sur&ce sweeping is a powerful tool for creating tube-like shapes, i.e., simulating blood 
vessels. The sweeping operation requires a smooth trajectory and cross-sectional shapes. To form a 

25 tube^Iike surface, a closed cross-sectional contour must be used, A cubic Bezier curve is used to 
represent the central trajectory or path. With y (t) representing any of G* paths, a local coordinate 
system (T (/), N(/), B(r)) can be defined along the curve (see, e.g.. Figure 5). This triplet (T (/), N(0, 
B(0), also known as a "Frenet frame", is the tangent, normal and bi-normal defined along the 
trajectory of the curve. Assuming r(t) is a contour function defined in the cross-sectional plane 

30 x>^rpendicular to the curve at a given point along the trajectory, the sweeping surface can be 
represented as 

r(/, 8) =Y(0+''ft^ (cos^N(0+sin^ B(/)), 
as described by Piegl and Tiller, In The NURBS Book, Springer, Berlin, 1995, where S is the cross- 
sectional angle and / e[Oj] is a parameter defined along the curve. A bi-cubic B^er form for the 
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sweeping tub e can therefore be developed using a tewRnr^TriHiir-t fip^HnTu e.g., Piegl and Tiller, 
1995, supra). 

To model bifurcation, flie same sweeping operation can be applied. In order to avoid self- 
intersection, only half of die tubular sur&ce can be used (Figure 6). This, however, leads to missing 
5 two triangular patches (front and back) at the joint Bifurcation modeling therefore requires triangular 
hole filling. An analytic approach described in Gregory and Zhou, Computer Aided Geometric 
Design 7J_: 391-4 10, 1994, can be used to fill triangular holes with given neighboring surfaces. To 
generate smoodi bifurcations, however, additional modifications of hole boundaries and hence the 
surrounding surfaces are desirable. The procedures for bifurcation modeling are summarized as 
10 follows: 

(i) A bifurcation is first generated by sweeping diree semi-tubular surfaces in bi-cubic 
Bezierform. • 

(ii) Two triangular holes are formed by three surrounding semi-tubular surfaces. Each hole 
is initially "filled" with three bi-cubic Bteier patches using the method described in 

15 Gregory and Zhou, 1 994, ^Mpra. 

(iii) The boundaries of the semi- tubular surfaces are changed to quintic Bezier form. The 
modifications are determined from the cross-boundary tangential continuity, twist- 
compatibility and unique existence of tangent planes at hole comers. 

(iv) Three semi-tubular surfaces are then degree-elevated into quintic B^ier patches and 
20 modified based on the new hole boundaries. The next row of control points of the hole 

boundaries are modified accordingly to ensure that the semi-tubular surfaces having 
cubic cross-boundary derivatives along the hole boundaries. 

(v) The vector- valued cross-boundary derivative in a quintic form along the hole 
boundaries is generated for the filling hole patches. 

25 (vi) The hole boundaries are split into two at the middle point of the parameter, so are flie 

associated vector-valued cross-boundary derivatives. Hie star-lines and their 
associated vector-valued cross-boundaiy derivatives are degree-elevated to quintic as 
well, 

(vii) Three final filling rectangular patches are generated based on the updated starlines, 
30 split hole boundaries, and the vector-valued cross-boundary derivatives along the star- 

lines and the split hole boundaries. The remaining 3x3 interior control points are 
determined by taking a Coons-Boolean sum approach as described in Ye, Computer 
Aided Design 27: 875-885; 1995. 
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From the segmented medical images, a central line model of a vas culature can be constructed. 

This model is represented in hierarchical structure consisting of vessel topology (using a parent-child 
relationship to represent the topological connectivity among a list of a vascular segments), vessel 
geometry (coordinates and radii), and vessel material property. The 3D model of the vessels is then 
5 leconstnicted based on the central line geometry. Visual smoothness is achieved by employing 
operations like sweeping and blending. A variational modeling approach is implemented for 
vasculature segments. An advantage of such method is that it provides flexibility in changing 3D 
structure. Where a pathology is identified and measured, a vascular model can be modified to 
account for the pathology. 

10 Preferably, deformable models are used to detect structures in images. Such models can be 

used to define a geometry which minimizes the energy of a simulated structure to account for 
topological change, e.g., due to factors such as blood flow dynamics and even interactions between 
the device and the body lumen or cavity. For exarhple, catheter tip shape deforms in a predictable 
marmer when straigihtened with a guide wire, when advancing through tortuous vessels, and when 

1 5 encountering vascular constraints such as lumen narrowing, branch point bifurcations, and ^e like. 
These events can be modeled using the simulation system. 

In one aspect, a deformation law is applied to a geometric model obtained ftom a hierarchical 
central line model to construct a 3D model. This 3D model can be used to model linear deformation, 
linear forces, non-liiiear deformation, and non-linear forces. The application of deformation laws to a 
20 geometric model is described in, for example, Wang, et al., 1999, supra, Mallaldi, et aL, J, 

Mathematical Imaging and Vision 6(7-3) 1 269-289, 1996; Caselles et al., Ntunerische mathematik 66: 
1-31, 1993; Osherand Sethian,y. Computational Physics 79: 12-49, 1988; Se^iait, In Level Set 
Methods, Cambridge University Press, Cambridge, England, 1996, Caselles et al.. Int. J. Vis. 22(l) i 
61-79; Kichenassamy et ?\,Proc, 5^^ Int. Conf. Computer Vision, pp. 810-815, 1995). 

25 In a further aspect, the system corrq>rises a knowledge base, relating to the physical and/or 

biological properties of a patient(s)* body cavity or lumens. Facts within this "vascular material 
knowledge base" can be derived in part from the geometric modeling arm of the simulation system as 
well as from public databases (e.g., such as PubMed®) (see, as shown in Figure 1). In one aspect, the 
property of elasticity can be established from the relationship between image density (determined 

30 from a volume image) of a portion of a body cavity or lumen and the stif&iess of a particular tissue. 
In another aspect, the diameter of a cavity/lumen can be determined. 

For example, plaque can be distinguished &om vessel walls by evaluating the image intensity 
of a volume image. Preferably, as volume images of body cavities or lumens are acquired from 
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patients having a disease, data relating to these images are provided to one or more of the knowledge 
base systems described above. Preferably, the knowledge base system(s) include data from images 
are obtained from patients having adierosclerosis, coronary vascular lesions, carotid bifurcation 
stenosis, carotid bifurcation stenosis, abdominal aortic aneurysms, peripheral vascular disease, 
5 cerebrovascular disease, cancer, trauma, and congenital malformations tiiat may cause or display 
vascular mtanifestations, and tiie like. 

' Quantitative measures of a pathology can be obtained. For example, a quantity module which 
is part of the system can be used to measure the size of a blockage (e.g., a plaque). 

In yet another aspect, at least one knowledge base comprises clinical information related to a 
10 specific patient for which a device is being designed. This database can include such demographic 

information as age, sex, drug history, medical history, medical billing information, and tiie like. This 
portion of the system can be encrypted so that while information can be continually added to other 
knowledge bases (e.g., by remote system useis), information within the patient-specific knowledge 
base cannot be tampered with. However, preferably, even information provided by remote system 
15 users will be stored in temporary data files until a system operator enables the system to accept the 
information. Information relating to populations of patients also can be stored for comparison with 
information relating to the specific patient 

Device Modeling System 

Preferably, the first display interface also displays a three-dimensional representation of the 
-20 medical device which is interfaced with the manikin. A simulation of &e device can be obtained 
from a database of images of stored devices (e.g., where these are known and/or commercially 
available) or from a simulation of a device, for example, as described in U.S. Provisional Application 
Serial No. 60/273,734, filed March 6, 2001. A volume scanned image of the device also can be 
generated using techniques similar to those described above. 

25 Preferably, a physical model is used to simulate a device based on the quantitative analysis of 

volume-rendered images, followed by a derivation of the geometry, topology, and physical properties 
of the device. Suitable medical devices which can be simulated include, but are not limited to: a 
catheter, guidewire, endoscope, laparoscope, bronchoscope, stoit, coil, balloon, a balloon-inflating 
device, a surgical tool, a vascular occlusion device, optical probe, a drug delivery device, and 

30 combinations thereof. The system is able to model the interactions of multiple devices with each 
other. For example, the system can model the simultaneous movements of a catheter, guidewire, 
theirapeutic device and the like. 
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Navigation Modeling System 

Preferably, the system simulates the movement of the simulated device within the simulated 
body cavity or lumen in real-time when a first user manipulates the medical device interfaced with the 
5 manikin. This can be realized by performing virtual device navigation inside a simulated body cavity 
or lumen using an incremental FEM engine. Such a simulation system is described in U.S. 
Provisional Application 60/273,733, filed March 6, 2001, the entirety of which is incoixx)rated by 
reference. For example, the embedded FEM engine of the sj^tem can provide a real-time simulation 
of catheter/guide- wire interactions with blood vessels. This enables the first user to develop the eye- 
10 hand coordination necessary to implement a particular interventional procedure. 

Depending on the realism or deformation accuracy required, the underlying assumption of 
physical modeling is adapted to simulate navigation and/or deployment of a particular medical device 
in a body cavity or lumen. 



15 portion of a patient's body cavity or lumen and determines fit between the geometry of the device 
(with or without functional and information attribute layers) and the geometry of the path. The 
system simulates the design of the device in stages; first providing a simulation based on optimal 
shape (e.g., using the device shape database), then optimal function, then optimal information 



Basic Models for Catheter Navigation 

In a preferred aspect, an incremental Finite Element Method (FEM) is applied to the analysis 
of catheter navigation. In this basic FEM analysis, blood vessels are assumed to be rigid circular 
tube-structures with varying radii, or with arbitrary cross-sections and deformabiiities. However, it is 

25 commonly accepted that because most of the blood vessels witfiin Hic human body are well stretched 
by surrounding muscles, the vasculature network is only minimally deformed by catheter navigation. 
Generally, also the tip of a guidewire or navigating catheter is very soft in order to prevent damage to 
a blood vessel when in contact The vessel wall is therefore relatively stiffer than the catheter itself 
and therefore, it is quite reasonable to simulate a rigid blood vessel wall when simulating navigation 

30 and/or deployment of a catheter. 



In a preferred aspect of the invention, the system simulates a path which represents at least a 



parameters. 



20 



Typical catheters/guidewires are generally cylindrical in shape, with curved or uncurved 
geometries (e.g., the cross-sections of catheters/guidewires are generally circular or ring-shaped). A 
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catheter/guidewire can be conceptualized as comprising a plurality of discrete 3D beam elements or 
segm^ts. Thus, they can be represented as an aggregate of multiple, flexible bodies connected to 
each other at nodes capable of simulating extension, compression, bending and torsion (Le., modes of 



5 multibody systems comprising a plurality of segments inside the rigid tube-structures (e.g., 
corresponding to blood vessels). 

In one aspect, a simulated catheter is subjected to a simulated force such as pushing, puUing, 
twisting, or combinations thereof. The movement of the catheter is represented as the sum of rigid- 
body displacements and deformations at each step with the deformations being relatively small 
10 compared to displacements. Generally, the elements of the catheter system are treated as rigid bodies 
first and then deformations are calculated for individual nodes of elements at their equilibrium 
position. 

A different principle can be employed which is based on a virtual work method or multibody 
dynamics method (MDM). The MDM models the kinematics (rigid body displacements) of a 
15 multibody system. In contrast, the FEM models the deformations of flexible bodies to analyze 

catheter navigation. Generally, the deformation of a medical device such as a catheter is relatively 
small relative to the rigidity of a lumen, such as a blood vessel and Hooke's law is applied. However, 
where the deformation of a lumen is large (for example, when biomaterials fail), nonlinear FEM 
should be applied. 

20 The Governing Equation 

Consider a catheter represented by multiple, discrete elements, each with a deformed 
configuration. This flexible multibody system is in dynamic equilibrium with applied forces (e.g., 
contact traction and internal forces) at time /. An exan^le of an internal force is a hyper-viscoelastic 
force in an isotropic constitution model (see, e.g.. The Biomedical Engineering Handbook, Editor-In 
25 Chief, Joseph D. Bronzino, CRC Press + IEEE Press, 1995). 

Two coordinate systems can be used to describe the movement and the deformations of the 
catheter system. The coordinate system XYZ is the inertial (global) reference frame and xyz is tiie 
body (local) reference frame at time /. 



deformation). Thus, catheter navigation can be considered as a sequence of movements of flexible 



The variational equations of motion of die catheter system are given as 
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^-Su^^S^-^iP^pU /->rfV--=-J^e^<f d-^dV C-1-) 

where 

S denotes the boundary surface, 

V denotes the volume of the catheter, 

u denotes the displacement vector of a catheter point at time t, 
S and^ denote the strain vector and stress vector, respecdvely, 

cr*^ denotes the j=esidual stress vector due to the accumuladon of the previous 
deformations which can be reduced through a process of energy release (shape 
recover), 

Su denotes the virtual displacement vector that is consistent with the constrain 
condidons, 

^£ denotes the comesponding virtual strain vector, 
p denotes the mass density of the catheter, 

/ denotes the body force vector. 

u denotes the acceleiadon vector (second order differentiation of variables with 
respect to time t), and 

T is the traction vector and can be expressed in terms of the external force applied at 
die boundary S and the contact force between the catheter and the walls of the 
vessels. 

The displacement vector can be defined as a summation of two terms, a term representing 
rigid t)ody displacement and a deformation term, in ord^ to derive an explicit equation which can be 
solved to simulate catheter movemenL Because rigid displacement of the cafiieter results in no strain 
5 or stress, the final two variational equations from (1) can be formulated as 

representing the movement of catheter/multibody system, with each element moving as a rigid object 
and 

10 whCTe u n denotes the rigid displacement; u / represents the deformation, T „ and T c represent the 
external force and contact force, respectively; and e /and a /represent die strain and stress 
corresponding to the deformation. It is assumed here that deformations are generally smaller than the 
rigid displacements. The deformation can then be assumed to be time independent during navigation. 
Because of this, the acceleration of deformation is not taken into consideration in tfiis formula. 

15 FEM Analysis 

Ihree-dimensional (3D) beam elements can be used to derive equations for FEM analysis of 
catheter movement. Equations (2) and (3), above, can be rewritten for a catheter widi TV elements, 
where 
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St =^]£-j;^^ - - 

represents the rigid body displacement of the beam elements and 



(4) 



represents &e deformations of the elements. Sn and Va denote the boundary and volume of the ntfi 
.5 element, respectively. 

Equations (4) and (5) can be solved using routine F£M methods to derive the matrix 
equations: 

Miy =7;+f, + 7 (6) 
10 ^=7;+7;-.M^+7+^** (7) 

providing a multidynamic analysis of rigid body displacements and for an F£M analysts of 
deformations for a simulated catheter. The rigid body displacement vector u is represented by three 
translations and three rotations at each FEM node, while the deformation vector U is also expressed 
15 in the same way, with a total of six degrees of freedom at an FEM node. Matrices M and K denote 
the global mass matrix and the global stif&iess matrix, respectively. Note that equations (6) and (7) 
are coupled to each other and can be solved using a semi-implicit iteration method (see, e.g., Rao, In 
The Finite Element Method in Engineerings 2"^ Edition, New York: Fergamon Press, 1989). 

Advanced Models 

20 In one aspect, vessels are not treated as rigid cylindrical structures but as deformable 

structures. This is particularly useful when modeling the interaction of catheterization devices, such 
as balloon and stents, with diseased arteries. These vessels vary in cross-sectional diameter due to the 
presence of a pathology, such as stenosis. Further, the deformation properties of diseased vessels are 
different from the deformation properties of normal vessels. Additionally, it may be desirable to 

25 model one or more physiological parameters such as the effect of blood flow on navigation of a 
medical device such as a catheter. Therefore, in one aspect, the simulation process uses a 
hemodynamic model describing the blood flow phenomenon- Preferably, the hemodynamic model 
accounts for the affects of such variables, as disease, age, obesity, and the like. 
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For a more realistic and accurate analysis of the mteiaction between a device and a patbology 
such as coronary stenosis, the device materials and vessels are treated as nonlinear. Blood flowing 
through the vessels is modeled by treating the blood as a homogeneous incompressible Newtonian 
fluid. The vessels and associated stenosis are modeled as layered structures, using a 3D space FEM 
S model. Physiological influences on a body cavity or lumen also can be considered in particular 
anatomic locations in wluch these influences may be important concerns. For example, when 
modeling coil deployment in the head, the system can take blood flow dynamics into accoimt, since a 
coil may be swept away if incorrectly placed. 

Preferably, FEM analysis occurs in real-time, in order to simulate real-time movement of a 
10 representation of a device while a iiser is manipulating a model device in the manikin. Such training 
is important to developing eye-hand coordination. Preferably, this is implemented using a fully 
nonlinear 3D finite element model combined with hemodynamic analysis. This requires extensive 
computer power. A multiprocessor ^stem with at least about four processors, with each processor 
having computing power equal to a 2 GHz Intel Pentium 4 CPU is one example of a system that 
1 5 might be used. 

Preferably, the system models interactions between the physiology and anatomy of a patient 
and one or more medical devices using physical modeling. Figure 16 is a block diagram illustrating 
the various steps in such a simulation process and how these interact with each otiier. Preferably, the 
system accounts for physiological responses to an intervention procedure. For example, in simulating 
20 damage to a vessel wall through excessive pushing of a device against the wall, the system also can 
simulate the effect such an action vnll have on blood pressure. Similarly, a simulation of a tumorous 
lesion in a blood vessel include modeling the effects of such a lesion on local mechanical properties 
of tissues. 

In a preferred aspect, the system includes a patient database comprising stored volume images 
25 and files relating to optical data obtained from body cavities and lumens of one or more patients. 

Preferably, flie database comprises data from a population of patients, such as a population of normal 
patients, a population of patients with coronary artery disease, peripheral vascular disease, stenosis, 
and the like. In one aspect, the database comprises vascular data from patients obtained Ihrougih one 
or more of a 3D Rotational XR scanner, an MRI scarmer, an MRA scanner, or any scanning device 
30 suitable for collecting scanned images for generating volume images. Data so obtained generally is 
segmented. By having two rounds of scaiming using the same scanning, but wifli only one round 
revealing the vasculature, segmented vessels can be obtained by subtracting images of the former 
scan from the corresponding images in the latter scan. 



Best Available Copy 



wo 02^)70980 , Page 34 of 71 



WO 02/070980 PCT/US02/06604 

33 



Fig ure 15 illustrates the creation of a physical mode l for a pa tiqifs vascul ature. Based upon 

segniented volume images, nodes are extracted, along wifli position and contour information. This 
data defines the central line of a vessel. Preferably, nodes are labeled according to the direction of 
blood flow as this aids in the deformaticn analysis process. After a 3D geometric model is 
5 reconstnicted from the medical images, using the geometric modeling system as described above, a 
deformation law is implemmted using F£M. 

Deployment Models 

In one aspect, in addition to modeling movement of a device through a body cavity or lumen, 
the system models one or more operations of the device. Such operations include, but are not limited 
10 to: a surgical procedure (e.g., such as removing, cutting, or repairing a tissue), inflation or deflation 
of a balloon, injection of a radioopaque material into the body cavity or lumen, and the like. 

In one aspect, the system models interactions between an angioplasty device and a blood 
vessel having one or more lesions. For example, the system can model the deployment of a balloon 
angioplasty device at a lesion site and can model inflation and deflation of the balloon at the site. 

15 Preferably, a visual simulation is implemented at the same time that a user manipulates a balloon- 
inflating device. In one aspect, the balloon-inflating device comprises a delivery mechanism for 
controlling delivery of fluid (e.g., such as air) to a balloon, a pressure sensor for monitoring pressure 
of a fluid delivered to the balloon, and an electrical pressure meter for reading pressure determined by 
the pressure sensor. The electrical pressure meter transmits a signal corresponding to a pressure value 

20 to the system processor and, in response, the system simulates deployment of the balloon both on the 
screen of the first user device and within the manikin. 

Injection of contrast medium also can be simulated. In one aspect, the first user (or a second 
user training the first user) can select an injection volume and rate from selectable options displayed 
on the first user interface (or second user interface). When these injection parameters are set, 

25 injection can be triggered using a simulated syringe or button on the control paneL The appearance of 
contrast filling the vessels (opacity) is modeled to reflect the rate of injection relative to the blood 
flow rate in the vessel. This provides a realistic impression of the injection. Preferably, diastolic and 
systolic flow pattern and washout of contrast through the vasculature also are modeled using the 
contrast injection function. Roadmapping capabilities additionally can be made available. The value 

30 of roadmapping is enhanced when it is combined with simultaneous 3D display of the vasculature. 
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Simulation Methods 

In one aspect, the system provides a haptic interface for providing an interventional 
radiologist with the sense of touch during pretreatment planning and training. The system can be 
used in preparation for catheterization procedures with cerebral vascular pathologies such as 
5 intracranial aneurisms, stenosis, arteriovenous malformations, and the like. Other simulations which 
can be performed using the system include, but are not limited to, angioplasty, catheter navigation in 
the abdominal aorta, peripheral intravenous catheterization, venipuncture, interventional cardiology, 
and the like. 

Figure 13 is a flow diagram illustrating a simulation according to one aspect of the invention. 
10 Preferably, the simulation is event driven, i.e., responding to discrete actions from one or more users. 
Computation of interactions between the body cavity or lumen and one or more medical devices is 
implemented by the FEM engine described above. FEM has been used widely to compute 
deformations under mechanical constraints in engineering; however, the instant invention applies 
FEM analysis to various physical models of blood vessels, blood flow dynamics and medical devices. 

15 Any input from users comes in the form of device manipulations, such as pushing, pulling, 

twisting a catheter, deploying a coil; inflating a balloon with a balloon catheter. Each of these inputs 
affects the finite element structure modeled by the system. By solving equations representing this 
structure through an iterative process as described above, an equilibrium state representing the 
interaction between tiie device and cavityAumen can be obtained. At the equilibrium state, the 

20 interacting devices and vessels are at dieir final deformable forms. The forces computed at each node 
of the interaction between a medical device/body cavity or lumen are then fed back to one or more 
users using simulated instruments interfacing with the system. System outputs include real-time 
representations of medical devices as they navigate and/or deploy in simulated body cavities or 
lumens. 

25 Real-time interaction is an important feature of the instant invention. The immersion of a 

user, and therefore, his or her ability to learn fix)m the simulation system, is directly linked to the 
bandwidth of various components of the simulation system. An acceptable bandwidth for visual 
display is in the range of about 20-60 Hz while an acceptable bandwidth for haptic display is in the 
range of about 300-1000 Hz (where 300 Hz is flie free hand gesture frequency). 

30 Two parameters that are particularly important for accurate perception by a user are latency 

and computation time. Latency measures the time between sensor acquisition (e.g., acquiring the 
position of a simulated medical device relative to a simulated body cavity/lumen) and system action 
(e.g., haptic rendering or force feedback). Computation time is that amount of time needed to 
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— — ^et«mine-the equilibrium state of a structure (e.g., a representation of a device and cavity/lumen) and 
to update the resulting models. 

There are several contributing causes of latency, including, but not limited to: time required 
for communication between input devices and the system processor, time for communication between 
5 the haptic display and the system processor, time for communication between the visual display (e.g., 
the 2D display) and the processor, time to compute collision detection, time for force feedback, and 
time for computing deformation models. Latency depends greatly on hardware and preferably the 
system comprises an at least about 16-bits bus for internal transmission within the embedded system 
(e.g., manikin interface), and a combination of serial and USB transmissions to create external links 
10 between simulated devices and the system processor. 

Realism is also important. Very often, real-time interaction and realism are correlated. For 
example. Figure 14 shows the relationship between response time and realism. Preferably, the 
simulation system according to the invention provides a visual feedback of 12-1 5 frames per second. 

Visual feedback is the most powerfril perception channel. The quality of visual rendering 
15 greatly influences user immersion and therefore the effectiveness of the simulation system. The 

system provides a library of 3D representations of medical devices and a library of 3D representations 
of normal or pathological patient anatomy. In one aspect, the user can select an entry point for 
insertion of a medical device into a simulated patient (e.g., at either a radial or femoral site) and in 
response to the user's manipulation, a haptic display will display the navigation of the device, through 
20 a selected body cavity or lumen. The system can be used to direct the navigation of the device 

towards or away from the heart, allowing a user to choose where navigation will begin to be tracked. 
The user interface can comprise direction buttons, enabling a user to maneuver the device in 
particular directions. 

Preferably, a current position of a navigating catheter within a body cavity/lumen is overlaid 
25 on a fluoroscopic view. When lequiied, especially, during a course of training, a user has the option 
to adjust a fluoroscopic image in a maimer similar to that employed in a CatliLab. For example, the 
user can zoom in/out of a particular view of a catheter-guidewire entry site for a closer look of the 
position and orientation of the catheter and guidewire. The user also can improve image quality by 
collimating the field of view using the shutter function of a simulated scarming device. The shutter 
30 function provides both lateral and vertical collimation to recreate a clinical setting: The contrast of 
the fluoroscopic image also can be modified. In addition, moving patient table control levers can 
change the position of the fluoroscopic field of view. These levers control the movement of die 
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image in the X and Y-axes as well as rotation and skew. They correspond to patient table control 
levers used in the clinical setting. 

As discussed above, the motion of one or more medical devices relative to a body cavity or 
lumen is tracked and updated simultaneously in fluoroscopic and 3D displays. The user can change 
5 the transparency of 3D rendered vessels to better compliment the geometry presented in the 

fluoroscopic view. The user also has the option of overlapping a representation of the device witii a 
representation of a surface-rendered vasculature. 3D-rendered vessels can be rotated in the x-, y-, and 
z-axis to better appreciate the position of the catheter/guidewire dp relative to the vascular anatomy. 
This feature is very useful in teaching selective vessel catheterization techniques. Besides 3D views, 
10 a fluoroscopic image can be viewed in stereo mode with the stereo image projected onto a split screen 
alongside the 2D image of a vessel. Users see the stereo virtual imagp on the screen of their user 
devices wearing a pair of stereo glasses (e.g., such as StereoGraphic Crystal Eyes). The stereo virtual 
objects consist of vessels, associated volume data, and the one or more interventional devices being 
navigated and/or deployed through the vessels. 

15 Besides providing stereo and 3D views of the catheterization process, the simulator has the 

capability of locating catheter tip position within multi-planar images of a body region. The system 
will highlight tip position relative to anatomical structures on the display of one or more user devices. 
Preferably, images of the tip are viewable in any of an axial, coronal, sagittal or combined mode, 
representing images of scanned anatomical stnictures (e.g., such as obtained from CT, MRI or actual 

20 cross-anatomical color images). It is an important aspect of the S3^tem to provide multi-functional 
views for teaching complex techniques, such as those used in interventional procedures. In one 
aspect, an endoscopic view is provided of the tip of the catheter within the enclosing vessels of the 
cardiovascular system. 

In another aspect, the system provides a haptic system for providing a neuroradiologist with 
25 the sense of touch during pretreatment planning and training. Preferably, the system comprises a 
database of angiographic information and a vascular model can be extracted from a database 
. comprising data relating to geometry and topology of the cerebral vasculature (e.g., using a 
knowledge base). Physical properties of vessels can be obtained from outside sources such as the 
literature or from various experts. A library of devices suitable for use with vessels in the brain is 
30 included in the system. In one aspect, the libraiy comprises data files relating to the geometry and/or 
physical properties of interventional devices, such as catheters, guidewires, stents, and coils. A 
device shape and material knowledge base can be provided as described in U.S. Provisional 
Application Serial No. 60/273,734, filed March 6, 2001. 
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In one as pect the system simulates an intr acranial blood vessel wifli an aneurysm 

Intracranial aneurysms are berry-like blisters in cerebral arteries tiiat are caused by a weakness of all 
vessel wall layers. If rupture of an aneurysm occurs, hemorrhage can cause serious damage to the 
brain. To prepare for an interventional pioceeding, a clear undexstanding of the anatomy surrounding 
5 an aneurysm is critical, such as die location and size of Ifae neck, or connection between the anemysm 
and vessel which feeds into it An aneurysm is usually clearly visible in volume and surface-rendered 
images. 

Preferably, the system displays a representation of the aneurysm and surrounding blood 
vessels and models the interactions of a simulated device being maneuvered in proximity to the 

10 aneurysm. In one aspect, the system is used to simulated tfie placement of one or more coils to treat 
the aneurysm. In one aspect, aneurysm coiling is simulated by navigating a medical device such as a 
coilwire to the aneurysm and replacing the wire with a coil. The coil is deployed by detaching the 
coil from the wire. Multiple coils can be deployed by repeating the detachment process with new coil 
wires. Each of diese actions can be simulated on the haptic display using the system according to the 

15 invention. 

In other aspect, a user can simulate other interventional procedures such as contrast fluid 
injection, inflation or deflation of a balloon, stent deployment, injection of a drug or therapeutic 
agent The system provides both haptic and visual feedback to the user to enable the user to develop 
the eye-hand coordination necessary to become skilled in these procedures. 

20 • As described above, the invention produces a highly realistic simulation environment for 

preplanning image-guided medical device delivery procedures and for training individuals who 
perform such procedures. In additi.on to its application in biomedical engineering, component parts 
of this invention may be applied to many other fields, for example, such as the field of 3D computer 
games. It should be apparent to those of ordinary skill in the art that variations, modifications, anid 

25 other implementations of what is described herein will occur to those of ordinary skill in the art 
without departing from die spirit and scope of the invention and the claims. 

All of the references, patents, and applications identified above, are expressly inooTxx>rated 
herein in their entireties. 
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Whatis^laimed-is; 

1. A system for simulating movemrat of a medical device in a body cavity or lumen of a patient, 
comprising: 

(a) a medical device comprising a first end for manipulation by a user and a 
portion comprising a second end insertable into a simulated body cavity or 
body lumen in a manikin; 

(b) a manikin comprising an interface for receiving the portion comprising the 
second end and for interfacing with a simulated body cavity or lumen within 
the manikin, wherein the interface comprises a directional force feediback 
mechanism for exerting a directional force on the medical device in response to 
a feedback signal received by the force feedback mechanism. 



2. The system according to claim 1, wherein the directional force feedback mechanism provides 
resistance to forward motion but enables free reverse motion in response to the feedback 
signal. 

3. The system according to claim 1, wherein the directional force feedback mechanism 
comprises a rolling element coupled to ^e portion of flie device comprising the second end 
and wherein an internal surface of the simulated cavity or lumen in the manikin comprises an 

^ oblique slot for receiving &e rolling elemait 

4. The system according to claim 3, wherein, in response to a feedback signal, forward 
movement of the second end causes the rolling element to be received by the slot, thereby 
causing resistance to further forward motion. 

5- The system according to claim 4, wherein a motor controls movement of the rolling element 

6. The system according to claim 1 , ftirtiier comprising a tactile feedback mechanism. 

7. The system according to claim 6, wherein the tactile feedback mechanism provides continuous 
vibrational feedback to a user holding the medical device. 

8. The system according to claim 8, wherein continuous vibrational feedback is provided through 
a continuously rotating motor in communication with the portion of the device comprising tfie 
second end. 
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>. The s ystem according to claim 1 , wherein a position of at least the second end of the medical 



device relative to the manikin is continuously tracked. 

10. The system according to claim 9, wherein the medical device comprises an encoder for 
tracking the translation of the device and an encoder for tracking the rotation of the deviccL 

1 1. The system according to claim 9, wherein the system further comprises a tracking unit 
comprising a light source, a signal processing circuit, and one or more optical sensors, 
wherein the tracking unit is placed within the interface in optical communication with tfie 
device when it is inserted into the cavity or lumen. 

12. The system according to claim 1 1 , wherein light firom the light source reflects on the device 
when inserted and wherein &e reflected light is received by the one or more optical sensors. 



13. The system according to claim 12, wherein changes in reflected light received by the one or 
more sensors is detected by the system, and wherein, in response to this detection, the system 
simulates movement of the device in real-time on the user display. 

14. The system according to claim 12, wherein two optical sensois are provided which are 
perpendicular to one another. 

15. The system according to claim 1 2, wherein the tracking unit is configured in the form of a rail 
along which the device can move. 

16. The system according to claim 10, wherein one or more additional medical devices 
comprising a first end for manipulation by a user and a portion con^nising a second end for 
insertion intp the simulated body cavity or body lumen, are inserted into the inter&ce and 
wherein the position of each medical device is independently monitored. 

17. The system according to claim 16, wherein the one or more medical devices are selected from 
the group consisting of a catheter, guidewire, endoscope, laparoscope, bronchoscope, stent, 
coil, balloon, a balloon- inflating device, a surgical tool, a vascular occlusion device, optical 
probe, a dmg delivery device, and combinations thereof. 
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18. The system accoiding to claim 1, further comprising a table for placing the manikin thereon, 
wherein the table comprises a processor connectable to Uie network. 

19. The system according to claim 18, wherein the system further comprises at least one first user 
device connectable to the network, the first user device comprising a first display interface for 
displaying a three-dimensional representation of a simulated body cavity or lumen of a 
patient 

20. The system according to claim 19, wherein the first display interface further displays a three- 
dimensional representation of a medical device corresponding to a medical device which is 
interfaced with fhe manikin and wherein the system simulates on the display the movement of 
the medical device within the simulated body cavity or lumen of the manikin in real-time 
when a first user manipulates the medical device interfaced with the manikin, 

20. The system according to claim 19, further comprising a simulated scanning display for 
displaying a two-dimensional image of the simulated body cavity or lumen. 

21. The system according to claim 20, wherein the simulated scanning display is part of a 
simulated scanning device. 

22. The system according to claim 21 , wherein the simulated scanning device is simulating an x- 
ray imaging system. 



23. The system according to claim 21, wherein the simulated scanning device and scanning 
display are coupled to a movable C-arm wi&in scanning distance of tihie manikin. 

24. The system according to claim 1, further comprising a re-configurable control panel for 
performing one or more of: image acquisition selection; image display; manipulating a table 
on which the manikin is placed; manipulating the position of a simulated scanning device 
relative to the manikin; and control of one or more shutter devices for limiting a field of view 
of a scanning device placed within scanning distance of the manikin. 

25. The system according to claim 1 or 20, further comprising a monitoring station, the 
monitoring station comprising a second user device connectable to the network- and 
comprising a second display interface for enabling a second user to monitor the movement of 
the medical device within the simulated body cavity or lumen. 
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30. 

15 31. 

32. 
33. 

20 

34. 

25 35. 



The system according to claim 25, wherein the second display inter&ce of the 
monitoring station displays selectable options enabling the second user to select 
or change one or more anatomical and/or physiological parameters of the 
simulated body cavity or lumen, and wherein the selection causes the three- 
dimensional image of the simulated body cavity or lumen displayed to the first 
user to change to reflect the changed anatomical and/or physiological parameters. 

The system according to claim 20, wherein the system is coimectable to a 
database of patient images and/or medical data. 

The system according to claim 25, wherein the system is connectable to a 
database of patient images and/or medical data. 

The system according to claim 27, wherein the patient images con^iise images of 
a body cavity or lumen from a patient affected by a pathology. 

The system according to claim 28, wherein the patient images con^rise images of 
a body cavity or lumen £rom a patient affected by a pathology. 

The system according to claim 21, further comprising at least one foot-activation 
switch for activating or collimating the simulated scanning device, hiiage display 
or table movement 

The system of claim 1, wherein said lumen is a blood vessel. 

The system according to claim 27, wherein the first user display interface 
provides access to the database and wherein, in response to accessing, the system 
displays an image and/or medical data on the first user display interfuse. 

The system according to claim 27, wherein the second user display interface 
provides access to the database and wherein, in response to accessing, the system 
displays an image and/or medical data on the second user display interface. 

The system according to claim 33, wherein the second nser display interface 
provides access to the database and wherein, in response to accessing, the system 
displays an image and/or medical data on the second user display interface. 
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36, The system according to claim 35, wherein the second user display interface provides a 
selectable option enabling a second user to display the image displayed on the second user 
display interface, on the first user's display interface. 

37. The system according to claim 1 , wherein die device is selected fiom the group consisting of a 
catheter, guidewire, endoscope* laparoscope, bronchoscope, stent, coil, balloon, a balloon- 
inflating device, a surgical tool, a vascular occlusion device, optical probe, a drug delivery 
device, and combinations thereof. 



38. A syringe for simulating fluid delivery, comprising: 

a housing defining a lumen comprising an op>ening for delivering a fluid; 

a pushing element for pushing the fluid through tihe opening; 

a £riction-producing element in communication with the pushing element; and 

a rnotor in communication with Hie friction-producing element and comprising a 
signal-receiving element, 

wherein the friction-producing element causes friction between the pushing element 
and a surface of the lumen of the housing upon activation by the motor in response to a signal 
received by the signal-receiving element 

39. The syringe according to claim 38, wherein the motor, when activated, causes motion of the 
friction-producing element, thereby causing the friction-producing element to contact the 
surface of the lumen of the housing, creating friction between the pushing element and flie 
surface of the lumen and resistance to the motion of the pushing element 

40. The syringe according to claim 38, wherein the friction-producing element comprises one or 
more rubber pads. 

41. The syringe according to claim 40, wherein each rubber pad is coupled to an arm whose 
movement is controlled by the motor. 

. 42. The syringe according to claim 4 1 , wherein each arm is coupled to flie motor through a gear 
attached to the motor. . 

43. The syringe according to claim 38, wherein the amount of friction produced by the friction- 
producing element is adjusted by controlling a rotation angle of the motor. 
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45. 



46. 
47. 

48. 

49. 

50. 
51. 



The systeni according to claiin 1, further comprising the syringe of claim 38, wherein op>ening 
of the syringe is connectable to a connecting piece having a first end for receiving fluid from 
the opening and a second end for delivering fluid to a simulated body cavity or body lumen in 
the oianikin. 

A balloon-inflating device for simulating deployment of a balloon within a body cavity or 
lumen of a patient, comprising: 

a delivery mechanism for controlling delivery of fluid duougih the balloon-inflating 
device to the balloon; 

a pressure sensor for monitoring pressure of a fluid delivered to the balloon by the 
balloon- inflating device; 

an electrical pressure meter for reading pressure determined by the pressure sensor, the 
electrical pressure meter being connectable to a processor and for transmitting a signal 
corresponding to a pressure value to Oie processor. 

The system isiccording to claim 1, further comprising the balloon-inflating device of claim 45. 

The system according to claim 20, wherein the system simulates deformation of the body 
cavity or lumen by the medical device. 

The system according to claim 20> wherein the system simulates an operation of a medical 
device selected from the group consisting of: a surgical procedure, inflation or deflation of a 
balloon, injection of a radioopaque material into the body cavity or lumen, and combinations 
thereof. 

The system according to claim 20, wherein the system simulates the movement of device 
within a blood vessel. 

The system according to claim 49, wherein the blood vessel is in the brain. 
The system according to claim 50, wherein the blood vessel is in the heart. 
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— 52 A^method-foT-simiUating4h&moyement.of^-medical-d of a 

patient, comprising: 

providing a medical device comprising a first end for manipulation by a user and a 
portion comprising a second end inserted into a simulated body cavity or body lumen in a 
manikin^ wherein the simulated body cavity or lumen in the manikin comprises a directional 
force feedback mechanism, and 

wherein, in response to a feedback signal, the directional force feedback mechanism 
creates resistance to forward motion of the medical device but allows free reverse motion. 

53. The method according to claim 52, further comprising: 

providing a system comprising: 

a processor in communication with the directional force feedback mechanism, 
the processor connectable to the network; and 

a first user device in communication with the processor, the first user device 
comprising a first display interface for displaying a representation of a body cavity or 
lumen; and for providing access to a database of three-dimensional images of body 
, cavities and lumens from a plurality of different patients; and 

enabling a user to select from the database a representation, wherein in 
response to die selection, the representation is displayed on the first display interface. 

54. The method according to claim 52, wherein the first display interface a displays a three- 
dimensional representation of the medical device and wherein the system simulates the 
movement of the medical device within the body cavity or lumen in real-time as a first user 
manipulates the medical device which is inter&ced with the manikin. 

55. The metfaod according to claim 52 or 53, further comprising providing a monitoring station 
comprising a second display interface in communication with the processor and flie first 
display interface, and wherein the second display interface provides a second user with access 
to the database. 

56. The method according to claim 54, wherein when a second user selects a r^resentation firora 
the database, the representation is displayed on bo^ the first and second display interface. 

57. The method according to claim 53, wherein the system simulates the deformation of a body 
cavity or lumen in response to movement of the medical device by the first user and displays 
the represeiitation of the deformation on the first display interface. 
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58. The method according to claim 53, wherein the medical device performs an (^ration on the 
simulated body cavity or lumen and the first display interface displays a simulation of the 
operation. 

59. The method according to claim 58, wherein the operation is inflation or deflation of a balloon 
within the simulated body cavity or lum«. 

60. The method according to claim 58, wherein the operation is injection of a radioopaque fluid 
within the body cavity or lumen. 

61. The method according to claim 52, wherein the device is selected from the group consisting of 
a catheter, guidewire, endoscope, laparoscope, bronchoscope, stent, coil, balloon, a balloon- 
inflating device, a surgical tool, a vascular occlusion device, an optical probe, a drug delivery 
device, and combinations thereof. 

62. The method according to claim 54, wherein a first user inserts one or more additional medical 
devices into the simulated body cavity or lumen, and the movement of each medical device is 
independently monitored. 

63. The method according to claim 52, wherein the simulated body cavity or lumen in the manikin 
fur&er comprises a tactile feedback mechanism, providing continuous vibrational feedback to 
a first user manipulating the device. 

64. A method for simulating fluid delivery into a body cavity or lumen of a patient comprising: 
(a) providing a syringe for simulating fluid delivery, the syringe corr^l^ising: 

a housing defining a lumen comprising an opening for delivmng a fluid; 
a pushing elCTient for pushing the fluid through ttie opening; 
a friction-producing element in communication with the pushing element; and 
a motor in communication with ^e frictioh<-producing element and comprising 
a signal-receiving element, 

wherein the friction-producing element causes friction betwecai the pushing 
element and a surface of tiie lumen in response to a signal received by the 
signal receiving element; and 
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(b) providing a signal, thereby causing friction between the pushing element and tfie 
lumen. 

65. A mesthod for simulating deployment of a balloon within a body cavity or lumen of a patient, 
comprising: 

(a) providing a balloon-inflating device, comprising: 

a delivery mechanism for controlling delivery of a fluid through flie balloon- 
inflating device to the balloon; 

a pressure sensor for monitoring pressure of a fluid delivered to the balloon by 
the balloon-inflating device; 

an electrical pressure meter for reading pressure determined by the pressure 
sensor and for transmitting a signal corresponding to a pressure value to a processor; 

(b) providing a system comprising: . 

a processor for receiving the signal, the processor connectable to the network; 

and 

a user device comprising an interface displaying a representation of flie balloon 
within a simulated body cavity or lumen; and 

(c) delivering the fluid to the balloon; wherein deployment of the balloon in response to 
the delivering is displayed on tfie user device, 

66. The method according to claim 65, wherein the fluid is air. 

67. The method according to claim 65, wherein the method is used to simulate balloon 
angioplasty. 

68. The method according to claim 65, further comprising providing the system according to 
claim 1, inserting a balloon catheter into ^e simulated body cavi^ or lumen to simulate 
navigating to a target region of the body, and simulating positioning the baUoon d^loyment 
device in proximity to the balloon catheter to inflate or deflate the balloon. 

69. The method according to claim 67, further comprising inserting a catheter and guidewire into 
the body cavity or lumen to navigate the balloon cavity to the target region. 



wo 02/070980 



Page 48 of 71 





PCTAJS02/06604 



WO 02/070980 



47 



t: 



70. The method according to claim 67, further comprising inserting a stent catheter to navigate to 
a target region and using the balloon to dqp>loy the stent, thereby simulating stent deploymmt 
in the body cavity or lumen. 

71. The me^od according to claim 68, furdier comprising inserting a catheter or guidewire into 
the body cavity or lumen to navigate the stent catheter to the target region. 

72. A method for simulating coil embolization in a body cavity or lumen of a patient, comprising: 



region of the body; 

providing the system according to claim 19, wherein the re-configurable control panel 
provides a selectable option for detaching die coil fiom the coil wire, and wherein selecting 
the selectable option triggers tiie release of the coil from the coil wire. 

73. The method according to claim 72, wherein an electrical current triggers release of the coil 
firom the coil wire. 



providing a catheter, guidewire and coil wire comprising a coil to navigate to a target 
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